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TITLE 
Multi-sensor evaluation of a printing process 
TECHNICAL FIELD 


The aspects and embodiments of the present disclosure relate to automated assessment of a 
printing process by a 3D bioprinter. More specifically, the aspects and embodiments of the 
present disclosure relate to systems and methods for automatically assessing the printing 


process of the 3D bioprinter by means of sensor devices and sensor data analysis. 
BACKGROUND 


During the past few years, the research and development activities related to 3-dimentional 
(3D)-bioprinting have exploded in number and many different approaches are currently being 
explored. Amongst many technical and design parameter considerations and implementations 
of a 3D bioprinting process is the ability to ensure a smooth and seamless flow of the printing 
process. To this end, an important step is to ascertain during the entire printing process that all 
the parameters and quality metrics are met by the process and by the end product i.e. the 
printed object. This includes the pre-print phase wherein e.g. adequate priming of a dispensing 
means such as a dispensing nozzle and/or a dispensing needle of the 3D-bioprinter configured 
to deliver the printing fluid to the printing bed is to be ensured. Further, during a run-time of 
the printing process, performing the act of printing of the desired structures is to be monitored 
and verified. Additionally, in the post-print phase, the printed objects are to be monitored for 


the integrity and be verified against expected print quality metrics. 


Conventionally, the process of verification of a printing process has been associated with a wide 
range of problems including unnecessary user involvement in the monitoring and verification 
procedure, inadequate calibration of the 3D-bioprinter in the pre-print phase, inadequate 
priming of the bioprinter nozzle, inaccurate assessment of print metrics during pre-print, run- 
time and post-print phases, increased amounts of material waste per print job or with 
continuation of the printing process, risks of erroneously allowing a faulty print job to continue 


beyond an irreparable state, etc. 
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Accordingly, for elevated comfort, safety and accuracy of a printing process by a 3D-bioprinter, 
there is a need for solutions in the art that are capable of monitoring and verification of the 


entire printing process in a robust and reliable manner. 
SUMMARY 


It is therefore an object of the present disclosure to provide a control system, a 3D bioprinter 
comprising such a control system, a method, a computer program carrier, and a computer 
program product which alleviate all or at least some of the drawbacks of presently known 


solutions. 


More specifically, it is an object of the present disclosure to alleviate problems related to 


automated monitoring and verification of a printing process by a 3D bioprinter. 


These objects are achieved by means of a control system, a 3D bioprinter comprising such a 
control system, a method, a computer program carrier, and a computer program product, as 
defined in the appended independent claims. The term exemplary is in the present context to 


be understood as serving as an instance, example or illustration. 


To monitor and analyze the printing process in a pre-print phase, in real time during the run- 
time of the print job and also in the post-print phase, several parameters such as dispensing 
nozzle and/or needle priming, print quality, print texture, post-print reactiveness, structural 
analysis, quality of final print, etc. are to be considered. Accordingly based on the result of 
such monitoring and analysis decisions may be formed to continue or stop an ongoing printing 
process, or to change process parameters of the 3D bioprinter, adapt the design of the desired 
print object, and the like. The analyzing may also be used for notifying or warning a user of the 
3D bioprinter for unforeseen events or if printing reaches a predefined warning situation e.g., 


a close-to-borders situation, or structure-out-of-limits situation on the print bed. 


According to a first aspect of the present disclosure, there is provided a method for assessing a 
printing process performed by a 3D bioprinter. The method comprises obtaining sensor data 
associated with a plurality of sensor devices configured to obtain sensor detections of the 
printing process, the sensor data comprising sensor data associated with each sensor device of 


the plurality of sensor devices. The method further comprises processing the obtained sensor 
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data. Wherein the step of processing the obtained sensor data comprises any one of analyzing 
each sensor data associated with each sensor device separately in order to create at least one 
separately analyzed sensor data point, and fusing the obtained sensor data of at least two 
sensor devices of the plurality of sensor devices in order to create one or more fused sensor 
data point(s) and analyzing each of the created one or more fused sensor data point(s) in 
order to create one or more analyzed-fused data point(s), and analyzing each sensor data 
associated with each sensor device separately in order to create at least one separately 
analyzed sensor data point and fusing the at least one separately analyzed sensor data point 
with at least one other separately analyzed sensor data point in order to create one or more 
fused-analyzed sensor data combination(s), and fusing the at least one separately analyzed 
sensor data point with at least one of the one or more analyzed-fused data point(s). 
Additionally, the method comprises determining a weighted component for each of the at 
least one separately analyzed sensor data point, and/or for the one or more analyzed-fused 
data point(s), and/or for the one or more fused-analyzed sensor data combination(s), and/or 
for the fused at least one separately analyzed sensor data point with the at least one of the 
one or more analyzed-fused data point(s) by means of an association weight function in order 
to create one or more weighted processed data set(s). Even further, the method comprises 
determining a sum vector based on the one or more weighted processed data set(s) by 
summing the one or more weighted processed data set(s). The method further comprises 
determining a state of the printing process by evaluating the determined sum vector against a 
print-process threshold value and generating a state-signal indicative of the state of the 
printing process based on the evaluation of the sum vector against the print-process threshold 


value. 


In some embodiments, the step of determining the state of the printing process may comprise 
determining if the determined sum vector fulfills at least one requirement of the printing 
process, wherein the print-process threshold value may comprise a predetermined threshold 


value for each of the at least one requirement of the printing process. 


In several embodiments, the method may further comprise controlling the printing process 
based on the generated state-signal. In various embodiments the printing process may 


comprise at least one of a pre-print phase, run-time print phase, and a post-print phase. 
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Accordingly, in several embodiments the method may further comprise controlling the 
printing process by forming a continuation or interruption decision for each one of the phases 


of the printing process based on the determined state of the printing process. 


According to various embodiments and aspects, the at least one requirement of the printing 
process may comprise any one of an allowable calibration of a nozzle of the 3D bioprinter, an 
allowable priming of the nozzle of the 3D bioprinter, an allowable state of a dispensing needle 
connected to the nozzle of the 3D bioprinter, an indicator of clogging of the nozzle of the 3D 
printer, an allowable point tip calibration of the nozzle of the 3D bioprinter, an allowable 
flowrate and/or a volume, and/or a temperature, and/or a viscosity of a printing fluid being 
introduced into the nozzle of the 3D bioprinter, and an allowable size and/or a volume of a 
hanging droplet of the printing fluid extruded out of the tip of the nozzle of the 3D bioprinter 


and/or at least one quality-of-print metric. 


In several embodiments and aspects the plurality of sensor devices may comprise any one of a 
camera, a thermal camera, a thermal sensor, a lidar, a radar, a laser, a Wheatstone bridge 
circuit arranged under a print bed of the 3D bioprinter, a pressure sensor, an ultrasound 


sensor, and a microwave radio frequency, RF, sensor. 


In several embodiments and aspects, the method may further comprise determining the 
weighted component, by means of the association weight function, by multiplying a 
predetermined weight value with each of the at least one separately analyzed sensor data 
point, and/or with the one or more analyzed-fused data point(s), and/or with the one or more 
fused-analyzed sensor data combination(s), and/or with the fused at least one separately 
analyzed sensor data point with the at least one of the one or more analyzed-fused data 


point(s). 


According to yet second aspect of the present disclosure there is provided a system for 
assessing a printing process by a 3D bioprinter. The system comprises processing circuitry 
configured to obtain sensor data associated with a plurality of sensor devices configured to 
obtain sensor detections of the printing process, the sensor data comprising sensor data 
associated with each sensor device of the plurality of sensor devices. Further, the processing 
circuitry is configured to process the obtained sensor data and analyze each sensor data 


associated with each sensor device separately in order to create at least one separately 
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analyzed sensor data point, and fuse the obtained sensor data of at least two sensor devices 
of the plurality of sensor devices in order to create one or more fused sensor data point(s) and 
analyze each of the created one or more fused sensor data point(s) in order to create one or 
more analyzed-fused data point(s), and analyze each sensor data associated with each sensor 
device separately in order to create at least one separately analyzed sensor data point and 
fuse the at least one separately analyzed sensor data point with at least one other separately 
analyzed sensor data point in order to create one or more fused-analyzed sensor data 
combination(s), and fuse the at least one separately analyzed sensor data point with at least 
one of the one or more analyzed-fused data point(s). The processing circuitry is further 
configured to determine a weighted component for each of the at least one separately 
analyzed sensor data point, and/or for the one or more analyzed-fused data point(s), and/or 
for the one or more fused-analyzed sensor data combination(s), and/or for the fused at least 
one separately analyzed sensor data point with the at least one of the one or more analyzed- 
fused data point(s) by means of an association weight function in order to create one or more 
weighted processed data set(s). In addition the processing circuitry is configured to determine 
a sum vector based on the one or more weighted processed data set(s) by summing the one 
or more weighted processed data set(s). further, the processing circuitry is configured to 
determine a state of the printing process by evaluating the determined sum vector against a 
print-process threshold value and generate a state-signal indicative of the state of the printing 


process based on the evaluation of the sum vector against the print-process threshold value. 


According to a further third aspect, there is provided a computer program carrier carrying one 
or more computer programs configured to be executed by one or more processors of a 
processing circuitry, the one or more programs comprising instructions for performing the 
method according to any one of the embodiments of the method herein, and wherein the 
computer program carrier is one of an electronic signal, optical signal, radio signal or a 


computer-readable storage medium. 


According to yet another fourth aspect, there is provided a computer program product 
comprising instructions which, when the program is executed by one or more processors of a 
processing circuitry, causes the processing circuitry to carry out the method according to any 


one of the embodiments of the method herein. 
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According to a fifth aspect, there is provided a 3D bioprinter comprising one or more print 
heads, each print head comprising a dispensing nozzle having an inlet for introduction of a 
printing fluid into the dispensing nozzle and an outlet for extruding the printing fluid from the 
outlet of the dispensing nozzle; and a system according to any one of embodiments of the 


system herein for assessing the printing process performed by the 3D bioprinter. 
Further embodiments of the different aspects are defined in the dependent claims. 


It is to be noted that all the embodiments, elements, features and advantages associated with 
the first aspect also analogously apply to the second, third, fourth and fifth aspects of the 


present disclosure. 


These and other features and advantages of the present disclosure will in the following be 


further clarified in the following detailed description. 
BRIEF DESCRIPTION OF THE DRAWINGS 


Further objects, features and advantages of embodiments of the disclosure will appear from 
the following detailed description, reference being made to the accompanying drawings. The 


drawings are not to scale. 


Fig. 1 is a schematic block diagrams illustrating various embodiments of a system for assessing 


a printing process according to the present disclosure; 


Fig. 2 is a schematic flowchart illustrating a method in accordance with several embodiments 


of the present disclosure; and 


Fig. 3 is a schematic front view illustration of a 3D bioprinter comprising the control system in 


accordance with an embodiment of the present disclosure. 
DETAILED DESCRIPTION 


Those skilled in the art will appreciate that the steps, services and functions explained herein 
may be implemented using individual hardware circuitry, using software functioning in 
conjunction with a programmed microprocessor or general purpose computer, using one or 
more Application Specific Integrated Circuits (ASICs) and/or using one or more Digital Signal 


Processors (DSPs). It will also be appreciated that when the present disclosure is described in 
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terms of a method, it may also be embodied in one or more processors and one or more 
memories coupled to the one or more processors, wherein the one or more memories store 
one or more programs that perform the steps, services and functions disclosed herein when 


executed by the one or more processors. 


In the following description of exemplary embodiments, the same reference numerals denote 


the same or similar components. 


Fig. 1 illustrates a schematic block diagram of a plurality of sensor devices 6a-6n, collectively 
referred to as a sensing unit 6. The sensor devices 6a-6n are adapted to acquire information 


from the printing process performed by a 3D bioprinter 100. 


In various aspects and embodiments, the sensing unit 6 comprises different types of sensor 
devices 6a-6n which may be any one of or any multi-sensor combination of imaging sensor 
devices such as a camera like a fixed-focused camera or an auto-focus camera, or other types 
of sensor devices such as a thermal camera, a thermal sensor, a lidar, a radar, a laser, a 
Wheatstone bridge circuit arranged under a print bed 101 of the 3D bioprinter 100, a pressure 
sensor, an ultrasound sensor, and a microwave radio frequency, RF, sensor, etc. It should be 
appreciated that the sensor devices 6a-6n may be physically collocated in the sensing unit 6 or 
in several embodiments be placed in different locations with respect to each other, yet being 
referred to as the sensing unit 6 collectively. The sensing unit 6 comprising the one or more 
sensor devices 6a-6n is configured to, based on the defined functionality of each sensor 
device, acquire information from various components, units, or objects involved in the 
printing process. For instance, the sensing unit 6 may be configured to acquire information 
from one or more units or modules of the 3D bioprinter such as position or elevation of one or 


more printheads 1 of the 3D bioprinter 100. 


Further, the sensing unit 6 is configured to acquire information from object under print 110 
which may be referred to as the print object 110 such as various parameters associated with 
the structural, mechanical, chemical, thermal, optical properties of the print object 110 during 
various phases of the printing process such as during the run-time phase or in post-print 
phase. Further, the sensing unit 6 may be configured to acquire information from a dispensing 
nozzle 2 of each printhead 1 such as various parameters of a printing fluid being present in 


and to be extruded from the dispensing nozzle 2. The nozzles and/or needles of the printhead 
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1 may be used interchangeably throughout this description. In some embodiments a nozzle 2 
may be of the type of a dispensing needle 2 or the nozzle may comprise a needle as a part of 


the nozzle. 


The printing fluid may be comprised in the printhead 1 e.g. stored in a container (not shown) 
of the printhead 1 and be supplied to the respective nozzles 2 of the one or more printheads 1 
to be extruded out of the nozzles 2 during various phases of the printing process such as 
priming and print job. The printing fluid may comprise any one of a bioink, a printing ink, a 
hydrogel, a chemical reagent, a cell medium, crosslinking agents, and a gel. Examples of bioink 
may comprise any one of collagen, CoIMA, GelMA, GelXA, alginate, pluronic, nanofibrillated 
cellulose, HAMA, Cellink, etc. In this description, when using printing fluid it should be 
appreciated that it encompasses any one, or any combination of the above examples of the 
printing fluid as well as embodiments where external objects may be present in the printing 
fluid to be printed. External objects present or mixed into the printing fluid may for instance 
be any one of at least one living cell, at least one bead such as spherical plastic beads, 
fluorescent beads used for tracing, etc., at least one organoid i.e. aggregates of living cells, and 
at least one spheroid i.e. spherically-shaped aggregates of living cells, etc. the sensing unit 6 
may also be configured to acquire information regarding the external objects present in the 
printing fluid, while the printing fluid being extruded from the nozzle 2, under the print job 
phase i.e. when the print object 110 is being constructed and/or after the print object 110 is 


partly or fully constructed. 


According to several embodiments and aspects, the sensing unit 6 is configured to obtain raw 
sensor measurements from the different components and objects as explained above and 
transmit the obtained raw sensor measurements to processing circuitry 11 adapted to process 
the raw sensor measurements of the plurality of sensor devices 6a-6n and obtain associated 
sensor data. The raw measurements may in some embodiments be pre-processed for instance 
go through steps of filtering, amplification, noise reduction etc. In some embodiments raw 
sensor data may be used directly. 10 some embodiments and examples a combination of pre- 
processed sensor data and raw sensor data may be used by the processing circuitry 11. 
Accordingly, it should be understood that parts of the described solution, particularly a system 
10 may be implemented either in the 3D bioprinter 100 itself, in a system located external the 


3D bioprinter 100, or in a combination of internal and external the 3D bioprinter; for instance 
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in an external server or remote control center (not shown) in communication with the 3D 
bioprinter 100. The solution may in several embodiments be fully or partly implemented on 


external networks 20 such as on a cloud platform 20. 


The system 10 which may also be referred to as a control system 10 or a control device 10 
comprises processing circuitry 11 configured to perform assessing of the printing process by 
the 3D bioprinter 100 in accordance with several aspects and embodiments of the present 
disclosure. The processing circuitry 11 is configured to obtain sensor data associated with the 
plurality of sensor devices 6a-6n configured to obtain sensor detections of the printing 
process, the sensor data comprising sensor data associated with each sensor device of the 
plurality of sensor devices. Several types of sensor devices including cameras, thermal 
cameras, thermal sensors, Lidars, Radars, lasers, Wheatstone bridge circuit under the print 


bed, pressure sensors, ultrasound, microwaves, etc. can be used for acquiring measurements. 


As shown in Fig. 1, the sensor measurements from the sensing unit 6 is provided to the system 
10 comprising processing circuitry 11. The processing circuitry 11 may be configured to at least 
partly pre-process the raw measurements and obtain sensor data associated with the plurality 
of the sensor devices 6a-6n. The obtained sensor data may be provided to a data processing 
unit 11a of the system 10. In several embodiments, the data processing unit 11a may be 
implemented as a part of or as a functionality of the processing circuitry 11 or be a separate 
data processing unit in communication with and subordinate to the processing circuitry 11. 
The data processing unit 11a may also be referred to as data fusion unit 11a. The processing 
circuitry 11 is configured to process the obtained sensor data from the sensing unit 6. The data 
processing according to several aspects and embodiments comprises analyzing each sensor 
data associated with each sensor device 6a-6n separately in order to create at least one 
separately analyzed sensor data point. The data processing also comprises fusing the obtained 
sensor data of at least two sensor devices 6a-6n of the plurality of sensor devices 6a-6n in 
order to create one or more fused sensor data point(s) and analyzing each of the created one 


or more fused sensor data point(s), thus creating analyzed-fused sensor data point(s). 


In the present context, by analyzed-fused data points(s) it is meant to convey that the sensor 
data of the at least two sensor devices is combined (fused) prior to the analysis step, thus it is 


the fused sensor data which is analyzed by the processing circuitry. 
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Further, the processing circuitry 11 is configured to fuse the at least one separately analyzed 
sensor data point with at least one other separately analyzed sensor data point in order to 
create one or more fused-analyzed sensor data combination(s). In the present context, by 
fused-analyzed data combination(s) it is meant to convey that one or more combination(s) 


(data fusions) of separately analyzed sensor data are created by the processing circuitry. 


Even further, the processing circuitry 11 is configured to fuse the at least one separately 
analyzed sensor data point with at least one of the one or more analyzed-fused data point(s) 
in order to create one or more fused at least one separately analyzed sensor data point with 


the at least one of the one or more analyzed-fused data point(s). 


In other words, the sensor data associated with each of the sensor devices may be used on its 
own and be analyzed separately or it can be combined with sensor data from other sensor 
devices to create fused sensor data points. For instance sensor data associated with one or 
more cameras may be fused with the sensor data from one or more radar sensor devices to 
create a data representation such as point-cloud representation or pooled depth map of the 
obtained sensor data for instance created by a data representation unit 11b or data 
representation functionality 11b of the processing circuitry 11 as shown in Fig. 1. Similar data 
representations may be created for any sensor data type obtained from various sensor 
devices. The created data representations may later be further analyzed by the processing 


circuitry for assessment of the printing process as will be explained in more detail below. 


Not only the obtained sensor data of at least two sensor devices may be fused but also the 
separately analyzed data points from single sensor devices may be fused together. This way 
different data types such as vision-based data types may be combined with pressure or 
temperature data types which cannot be directly fused together however, but can be analyzed 


separately and processed in fused analyzed sensor data combination(s). 


Thus, the data fusion may be combination of two different set of data types being fused and 
analysed (like a signal and an image analyzed separately) and/or may be combination of data 
obtained from two different types of sensors to create a new set of fused data points and 
analyzing the fused data points (e.g. 3D mapping from a Lidar sensor and image data from a 


camera). 
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In several aspects and embodiments, not only the obtained sensor data acquired from online 
sensor measurements by the sensing unit 6 is used in the data processing but also offline 
sensor data from previous sensors measurements which for instance may be stored on a cloud 
network may be used to create the analyzed and fused data points. It should be noted that in 
the rest of this description when referring to a data point or data set as fused data point or 
fused data set it is to be construed as any of the different ways of combining and analyzing 


data obtained from the printing process regardless of the data type. 


In some exemplary embodiments wherein imaging sensor devices such as cameras are used as 
primary sensor devices, raw image measurements may be obtained from various objects and 
components e.g. the nozzle 2 or the print object 110 and the raw image data obtained by the 
imaging sensor device may be processed by means of an image processing algorithm. Vision- 
based data fusion techniques such as SLAM (Simultaneous localization and mapping) can then 


be applied to the obtained associated sensor data from the imaging sensor devices. 


Accordingly, as realized by the present inventor, the proposed solution makes it possible that 
data retrieved from multiple sensor devices, regardless of the type of the sensor device, work 
in synchrony and interpretation of data will be aligned and equal in the analysis part of the 


system. 


When the associated sensor data is processed by the system 10, and the data representation 
is created, the processing circuitry 11 is configured to determine a weighted component for 
each of the at least one separately analyzed and/or fused data by means of an association 
weight function in order to create one or more weighted processed data set(s). The fused data 
as mentioned earlier comprises the one or more analyzed-fused data point(s) and/or one or 
more fused-analyzed sensor data combination(s). The processing circuitry 11 is further 
configured to determine a sum vector, i.e. addition vector, based on the one or more 
weighted processed data set(s) by summing the one or more weighted processed data set(s). 
in some embodiments the weighted sum might be predefined. Accordingly, the processing 
circuitry may be configured to determine the weighted component, by means of the 
association weight function, by multiplying a predetermined weight value with each of the at 
least one analyzed and/or one or more analyzed-fused data point(s) and/or one or more 


fused-analyzed sensor data combination(s). 


10 


15 


20 


25 


30 


WO 2023/224528 PCT/SE2023/050389 
12 
By creating such a vectorized representation of the sensor data, the processing circuitry is 
adapted to determine a state of the printing process by evaluating the determined sum vector 
against a print-process threshold value and generate a state-signal indicative of the state of 
the printing process based on the evaluation of the sum vector against the print-process 
threshold value. The processing circuitry may further be configured to control the printing 


process based on the generated state-signal. 


In several aspect and embodiments, the processing circuitry may be further configured to 
determine the state of the printing process by determining if the determined sum vector 
fulfills at least one requirement of the printing process, wherein the print-process threshold 
value comprises a predetermined threshold value for each of the at least one requirement of 
the printing process. In other words, the processing circuitry is configured to compare the 
vectorized representation of the sensor data from the printing process with an assessment 
criteria which comprises requirements of the printing process to be fulfilled. Each of these 
requirements are accordingly associated with a respective predetermined threshold value. 
Thus, the processing circuitry 11, is adapted to compare the sum vector with each of the 
predetermined threshold values in order to determine if the printing processes fulfills the 


requirements. 


As also mentioned earlier, the printing process may comprise at least one of a pre-print phase, 
run-time print phase (i.e. the print job), and a post-print phase. In several embodiments and 
aspects, the at least one requirement of the printing process may comprise any one of an 
allowable calibration of a nozzle of the 3D bioprinter, an allowable priming of the nozzle of the 
3D bioprinter, an allowable state of a dispensing needle connected to the nozzle of the 3D 
bioprinter, an indicator of clogging of the nozzle of the 3D printer, an allowable point tip 
calibration of the nozzle of the 3D bioprinter, an allowable flowrate and/or a volume, and/or a 
temperature, and/or a viscosity of a printing fluid being introduced into the nozzle of the 3D 
bioprinter, and an allowable size and/or a volume of a hanging droplet of the printing fluid 
extruded out of the tip of the nozzle of the 3D bioprinter. Even further, in some embodiments 
the at least one requirement of the printing process may further comprise at least one quality- 
of-print metric for the print object 110 during the run-time print phase and/or in the post- 


print phase. Various parameters such as print quality, print texture, post-print reactiveness, 
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structural analysis, print uniformity, continuity, and regularity, air bubbles in bioink, etc. of the 


print object 110 may be comprised in the quality-of-print metrics. 


In several embodiments and aspects the processing circuitry may be further configured to 
control the printing process by forming a continuation or interruption decision for each one of 
the phases of the printing process based on the determined state of the printing process. For 
instance, if during the print job the system 10 determines that an error has occurred in the 
printing process and the print object 110 does not meet the requirements of the quality-of- 
print metrics or if there have been irregularities in the position, volume, viscosity, temperature 
or any other parameters of the printing fluid, the system 10 may terminate the print job and 
further alert the user of the 3D bioprinter of the unforeseen event e.g. by displaying error 
signals on a display 102 of the 3D bioprinter 100 or by producing sound alarm signals. 
Similarly, if at a post-print phase, the system 10 determines that the finished print object 110 
does not meet the quality-of-print metrics, the system 10 may terminate any subsequent print 
jobs based on the current print parameters and also alert the user to reevaluate and/or 


update the parameters of the printing process. 


Fig. 2 shows a flow chart of a method 300 according to various embodiments and aspects of 
the present disclosure for assessing a printing process by a 3D bioprinter 100. All the 
elements, features, and advantages explained in relation to the other aspects also apply 


analogously to this aspect of the present disclosure. 


The method comprises obtaining 301 sensor data associated with a plurality of sensor devices 
6a-6n configured to obtain sensor detections of the printing process, the sensor data 


comprising sensor data associated with each sensor device of the plurality of sensor devices. 


The method further comprises processing 303 the obtained sensor data, wherein the 
processing comprises any one of analyzing 305 each sensor data associated with each sensor 
device separately in order to create at least one separately analyzed sensor data point, and 
fusing 307 the obtained sensor data of at least two sensor devices of the plurality of sensor 
devices in order to create one or more fused sensor data point(s), and analyzing 309 each of 
the created one or more fused sensor data point(s) in order to create one or more analyzed- 
fused data point(s), and analyzing each sensor data associated with each sensor device 


separately in order to create at least one separately analyzed sensor data point and fusing 311 
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the at least one separately analyzed sensor data point from step 305 with at least one other 
separately analyzed sensor data point from step 305 in order to create one or more fused- 
analyzed sensor data combination(s)and fusing 312 the at least one separately analyzed 
sensor data point with at least one of the one or more analyzed-fused data point(s) in order to 
create one or more fused at least one separately analyzed sensor data point with the at least 
one of the one or more analyzed-fused data point(s). By the fused at least one separately 
analyzed sensor data point with the at least one of the one or more analyzed-fused data 
point(s) in the present context it is meant that the outcome of one or more of the at least one 
separately analyzed sensor data points may be fused with the outcome of the at least one of 
the one or more created analyzed-fused data point(s) in order to generate a new dimension of 


fused data sets. 


When the obtained sensor data is processed by the processing circuitry 11, the analyzed data 
points and/or created data representations are subjected to further processing by the 
processing circuitry 11 using simple logic and/or neural networks. Thus the method 300 
further comprises determining 313 a weighted component for each of the at least one 
separately analyzed sensor data point and/or one or more analyzed-fused data point(s) and/or 
one or more fused-analyzed sensor data combination(s) and/or the fused at least one 
separately analyzed sensor data point with the at least one of the one or more analyzed-fused 
data point(s) by means of an association weight function in order to create one or more 
weighted processed data set(s). After creating the weighted components, the method further 
comprises determining 315 a sum vector based on the one or more weighted processed data 
set(s) by summing the one or more weighted processed data set(s). Further, the method 
comprises determining 317 a state of the printing process by evaluating 318 the determined 
sum vector against a print-process threshold value. When the vectorized sensor data is 
produced and evaluated against the print-process threshold value i.e. compared with the 
print-process threshold value, the method further comprises generating 319 a state-signal 
indicative of the state of the printing process based on the evaluation of the addition vector 
against the print-process threshold value. In other words, the result of the comparison of the 
sum vector with the print-process threshold value generates an output which is used to assess 


the printing process. 
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In some embodiments, the method 300 may further comprise determining 317 the state of 
the printing process by determining 321 if the determined sum vector fulfills at least one 
requirement 200 of the printing process, wherein the print-process threshold value comprises 
a predetermined threshold value for each of the at least one requirement of the printing 
process. Thus, the sum vector is compared with all of the print-process threshold values of the 
one or more requirement(s) of the printing process to determining the state of the printing 


process i.e. assessing the printing process. 


In several embodiments the method may further comprise controlling 323 the printing process 


based on the generated state-signal. 


In various embodiments, the printing process may comprise at least one of a pre-print phase, 


run-time print phase, and a post-print phase. 


In some embodiments the method may comprise controlling 323 the printing process by 
forming 325 a continuation or interruption decision which may be a binary decision for each 
one of the phases of the printing process based on the determined state of the printing 


process. 


In several embodiments, the at least one requirement of the printing process may comprise 
any one of an allowable calibration of a nozzle of the 3D bioprinter, an allowable priming of 
the nozzle of the 3D bioprinter, an allowable state of a dispensing needle connected to the 
nozzle of the 3D bioprinter, an indicator of clogging of the nozzle of the 3D printer, an 
allowable point tip calibration of the nozzle of the 3D bioprinter, an allowable flowrate and/or 
a volume, and/or a temperature, and/or a viscosity of a printing fluid being introduced into 
the nozzle of the 3D bioprinter, and an allowable size and/or a volume of a hanging droplet of 
the printing fluid extruded out of the tip of the nozzle of the 3D bioprinter and/or at least one 


quality-of-print metric. 


In several embodiments, the plurality of sensor devices may comprise any one of a camera, a 
thermal camera, a thermal sensor, a lidar, a radar, a laser, a Wheatstone bridge circuit 
arranged under a print bed of the 3D bioprinter, a pressure sensor, an ultrasound sensor, and 


a microwave radio frequency, RF, sensor. 
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In various embodiments, the method may further comprise determining 313 the weighted 
component, by means of the association weight function, by multiplying 327 a predetermined 
weight value with each of the at least one separately analyzed sensor data point and/or one or 
more analyzed-fused data point(s) and/or one or more fused-analyzed sensor data 
combination(s) and/or the fused at least one separately analyzed sensor data point with the at 


least one of the one or more analyzed-fused data point(s). 


Executable instructions for performing the above functions and features of the embodiments 
of the methods may, optionally, be included in a non-transitory computer-readable storage 
medium or other computer program product configured for execution by one or more 


processors of the processing circuitry. 


More specifically, there is provided a computer program carrier carrying one or more computer 
programs configured to be executed by one or more processors of a processing circuitry, the 
one or more programs comprising instructions for performing any one of the embodiments of 
the method 300 according to this disclosure. The computer program carrier may be one of an 


electronic signal, optical signal, radio signal or a computer-readable storage medium. 


Even further, there is provided a computer program product comprising instructions which, 
when the program is executed by one or more processors of a processing circuitry, causes the 
processing circuitry to carry out any one of the embodiments of the method 300 according to 


this disclosure. 


Fig. 3 is a schematic front view illustration of a 3D bioprinter 100 according to several aspects 
and embodiments of the present disclosure. The 3D bioprinter comprises one or more 
printheads 1, wherein each printhead 1 comprising a dispensing nozzle 2 having an inlet for 
introduction of a printing fluid into the dispensing nozzle and an outlet for extruding the 
printing fluid from the outlet of the dispensing nozzle 2. The 3D bioprinter 100 may comprise 
a sensing unit 6 comprising one or more sensor devices 6a-6n configured to obtain various 
types of sensor detection measurements of the dispensing nozzles, printing fluid, the print bed 
101 of the 3D bioprinter, the print object 110, etc. during various phases of the printing 
process i.e. pre-print phase, run-time phase, post-print phase. In some embodiments, the 3D 
bioprinter 100 may be provided with the sensing unit 6 as a separate entity being in 


communication with the 3D bioprinter 100. The 3D bioprinter also comprises a system 10 
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configured to perform any one of the embodiments of the method 300 according to this 


disclosure for assessment of the printing process performed by the 3D bioprinter. 


The control system 10 may further comprise one or more processors 11, a memory 8, a sensor 
interface 13 and a communication interface 14. The processor(s) 11 may also be referred to as 
a control circuit 11 or control circuitry 11 or processing circuitry 11. The control circuitry 11 is 
configured to execute instructions stored in the memory 8 to perform a method 300 for 
assessing the printing process according to any one of the embodiments disclosed herein. The 
memory 8 of the control system 10 can include one or more (non-transitory) computer- 
readable storage mediums, for storing computer-executable instructions, which, when 
executed by one or more computer processors 11, for example, can cause the computer 
processors 11 to perform the techniques described herein. The memory 8 optionally includes 
high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access 
solid-state memory devices; and optionally includes non-volatile memory, such as one or more 
magnetic disk storage devices, optical disk storage devices, flash memory devices, or other 


non-volatile solid-state storage devices. 


Further, the 3D bioprinter 100 may be connected to external network(s) 20 via for instance a 
wireless link or communication interface 14 via various technologies such as cellular long 
range or short range such as Wireless Local Area (LAN), WiFi, etc. communication 


technologies. 


The present disclosure has been presented above with reference to specific embodiments. 
However, other embodiments than the above described are possible and within the scope of 
the disclosure. Different method steps than those described above, performing the method by 


hardware or software, may be provided within the scope of the disclosure. 


For instance, according to an exemplary embodiment a cloud computing system 20 can be 
configured to perform any one of or any combination of the embodiments of the method 300 
presented herein. The cloud computing system may comprise distributed cloud computing 
resources that jointly perform the methods presented herein under control of one or more 


computer program products. 


10 


15 


20 


25 


30 


WO 2023/224528 PCT/SE2023/050389 
18 

The processor(s) 11 (associated with the control system 10) may be or include any number of 
hardware components for conducting data or signal processing or for executing computer 
code stored in memory 8. The system 10 may have an associated memory 8, and the memory 
8 may be one or more devices for storing data and/or computer code for completing or 
facilitating the various methods described in the present description. The memory may 
include volatile memory or non-volatile memory. The memory 8 may include database 
components, object code components, script components, or any other type of information 
structure for supporting the various activities of the present description. According to an 
exemplary embodiment, any distributed or local memory device may be utilized with the 
systems and methods of this description. According to an exemplary embodiment the memory 
8 is communicably connected to the processor 11 (e.g., via a circuit or any other wired, 
wireless, or network connection) and includes computer code for executing one or more of 


processes described herein. 


It should be appreciated that the 3D bioprinter 100 may further comprise a sensor interface 
13 which may also provide the possibility to acquire sensor data directly or via dedicated 
sensor control circuitry. The sensor devices 6a-6n such as imaging sensor devices e.g. camera 
sensors may communicate with the control system 10 using a local network setup, Ethernet, 


optical fibres, and so on. 


As used herein, the term "if" may be construed to mean "when or “upon” or "in response to 
determining or "in response to detecting" depending on the context. Similarly, the phrase "if it 
is determined' or ^when it is determined" may be construed to mean "upon determining or 
"in response to determining" or "upon detecting and identifying occurrence of an event" or 
"in response to detecting occurrence of an event" depending on the context. The term 
"obtaining" is herein to be interpreted broadly and encompasses receiving, retrieving, 
collecting, acquiring, and so forth directly and/or indirectly between two entities configured to 


be in communication with each other or with other external entities. 


Accordingly, it should be understood that parts of the described methods and systems may be 
implemented either in the 3D bioprinter 100, in a system located external the 3D bioprinter 
100, or in a combination of internal and external the 3D printer; for instance in a server in 


communication with the 3D printer, e.g. a so called cloud solution 20. For instance, sensor 
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data may be sent to an external system (not shown) and that system performs the steps of the 
methods herein. The different features and steps of the embodiments may be combined in 


other combinations than those described. 


It should be noted that the word "comprising" does not exclude the presence of other 
elements or steps than those listed and the words "a" or "an" preceding an element do not 
exclude the presence of a plurality of such elements. It should further be noted that any 
reference signs do not limit the scope of the claims, that the disclosure may be at least in part 


|^ 


implemented by means of both hardware and software, and that several "means" or "units" 


may be represented by the same item of hardware. 


Although the figures may show a specific order of method steps, the order of the steps may 
differ from what is depicted. In addition, two or more steps may be performed concurrently or 
with partial concurrence. Such variation will depend on the software and hardware systems 
chosen and on designer choice. All such variations are within the scope of the disclosure. 
Likewise, software implementations could be accomplished with standard programming 
techniques with rule-based logic and other logic to accomplish the various connection steps, 
processing steps, comparison steps and decision steps. The above mentioned and described 
embodiments are only given as examples and should not be limiting to the present disclosure. 
Other solutions, uses, objectives, and functions within the scope of the disclosure as claimed 
in the below described patent embodiments should be apparent for the person skilled in the 


art. 


10 


15 


20 


WO 2023/224528 PCT/SE2023/050389 


20 
CLAIMS 


1. A method for assessing a printing process performed by a 3D bioprinter, the 


method comprising: 


obtaining sensor data associated with a plurality of sensor devices configured to obtain 
sensor detections of the printing process, the sensor data comprising sensor data associated 


with each sensor device of the plurality of sensor devices; 


processing the obtained sensor data, wherein the processing comprises any one of: 
analyzing each sensor data associated with each sensor device separately in order to create at 


least one separately analyzed sensor data point, and 


fusing the obtained sensor data of at least two sensor devices of the plurality of sensor 
devices in order to create one or more fused sensor data point(s) and analyzing each of the 
created one or more fused sensor data point(s) in order to create one or more analyzed-fused 


data point(s), and 


analyzing each sensor data associated with each sensor device separately in order to 
create at least one separately analyzed sensor data point and fusing the at least one separately 
analyzed sensor data point with at least one other separately analyzed sensor data point in 


order to create one or more fused-analyzed sensor data combination(s); and 


fusing the at least one separately analyzed sensor data point with at least one of the one 


or more analyzed-fused data point(s); 
determining a weighted component for each of: 
the at least one separately analyzed sensor data point, and/or 
the one or more analyzed-fused data point(s), and/or 


the one or more fused-analyzed sensor data combination(s), and/or 


10 


15 


20 


25 


WO 2023/224528 PCT/SE2023/050389 
21 
the fused at least one separately analyzed sensor data point with the at least one of the 
one or more analyzed-fused data point(s) by means of an association weight function in order 


to create one or more weighted processed data set(s); 


determining a sum vector based on the one or more weighted processed data set(s) by 


summing the one or more weighted processed data set(s); 


determining a state of the printing process by evaluating the determined sum vector 


against a print-process threshold value; and 


generating a state-signal indicative of the state of the printing process based on the 


evaluation of the sum vector against the print-process threshold value. 


2. The method according to claim 1, wherein determining the state of the printing 
process comprises determining if the determined sum vector fulfills at least one requirement of 
the printing process, wherein the print-process threshold value comprises a predetermined 


threshold value for each of the at least one requirement of the printing process. 


3. The method according to any one of claims 1 or 2, wherein the method further 


comprises: 
controlling the printing process based on the generated state-signal. 


4. The method according to any one of claims 1 — 3, wherein the printing process 


comprises at least one of a pre-print phase, run-time print phase, and a post-print phase. 
5, The method according to claim 4, wherein the method further comprising: 


controlling the printing process by forming a continuation or interruption decision for each 


one of the phases of the printing process based on the determined state of the printing process. 


6. The method according to any one of claims 2 — 5, wherein the at least one 
requirement of the printing process comprises any one of an allowable calibration of a nozzle 
of the 3D bioprinter, an allowable priming of the nozzle of the 3D bioprinter, an allowable state 
of a dispensing needle connected to the nozzle of the 3D bioprinter, an indicator of clogging of 
the nozzle of the 3D printer, an allowable point tip calibration of the nozzle of the 3D bioprinter, 


an allowable flowrate and/or a volume, and/or a temperature, and/or a viscosity of a printing 
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fluid being introduced into the nozzle of the 3D bioprinter, and an allowable size and/or a 
volume of a hanging droplet of the printing fluid extruded out of the tip of the nozzle of the 3D 


bioprinter and/or at least one quality-of-print metric. 


7. The method according to any of the preceding claims, wherein the plurality of 
sensor devices comprise any one of a camera, a thermal camera, a thermal sensor, a lidar, a 
radar, a laser, a Wheatstone bridge circuit arranged under a print bed of the 3D bioprinter, a 


pressure sensor, an ultrasound sensor, and a microwave radio frequency, RF, sensor. 


8. The method according to any of the preceding claims, wherein the method 


further comprises: 


determining the weighted component, by means of the association weight function, by 


multiplying a predetermined weight value with each of the: 
at least one separately analyzed sensor data point, and/or 
the one or more analyzed-fused data point(s), and/or 
the one or more fused-analyzed sensor data combination(s), and/or 


the fused at least one separately analyzed sensor data point with the at least one of the 


one or more analyzed-fused data point(s). 


9. A computer program carrier carrying one or more computer programs 
configured to be executed by one or more processors of a processing circuitry, the one or more 
programs comprising instructions for performing the method according to any one of claims 1 
— 8, and wherein the computer program carrier is one of an electronic signal, optical signal, radio 


signal or a computer-readable storage medium. 


10. A computer program product comprising instructions which, when the program 
is executed by one or more processors of a processing circuitry, causes the processing circuitry 


to carry out the method according to any one of claims 1 — 8. 


11. A system for assessing a printing process by a 3D bioprinter, the system 


comprising processing circuitry configured to: 
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obtain sensor data associated with a plurality of sensor devices configured to obtain 
sensor detections of the printing process, the sensor data comprising sensor data associated 


with each sensor device of the plurality of sensor devices; 
process the obtained sensor data, wherein the processing circuitry is configured to: 


analyze each sensor data associated with each sensor device separately in order to 


create at least one separately analyzed sensor data point, and 


fuse the obtained sensor data of at least two sensor devices of the plurality of sensor 
devices in order to create one or more fused sensor data point(s) and analyze each of the 
created one or more fused sensor data point(s) in order to create one or more analyzed-fused 


data point(s), and 


analyze each sensor data associated with each sensor device separately in order to 
create at least one separately analyzed sensor data point and fuse the at least one separately 
analyzed sensor data point with at least one other separately analyzed sensor data point in 


order to create one or more fused-analyzed sensor data combination(s); and 


fuse the at least one separately analyzed sensor data point with at least one of the one 


or more analyzed-fused data point(s); wherein the processing circuitry is further configured to: 


determine a weighted component for each of the at least one separately analyzed 


sensor data point, and/or 
the one or more analyzed-fused data point(s), and/or 
the one or more fused-analyzed sensor data combination(s), and/or 


the fused at least one separately analyzed sensor data point with the at least one of the 
one or more analyzed-fused data point(s) by means of an association weight function in order 


to create one or more weighted processed data set(s); 


determine a sum vector based on the one or more weighted processed data set(s) by 


summing the one or more weighted processed data set(s); 


determine a state of the printing process by evaluating the determined sum vector 


against a print-process threshold value; and 
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generate a state-signal indicative of the state of the printing process based on the 


evaluation of the sum vector against the print-process threshold value. 


12. The system according to claim 11, wherein the processing circuitry is further 
configured to: 
determine the state of the printing process by determining if the determined sum vector 
fulfills at least one requirement of the printing process, wherein the print-process threshold 
value comprises a predetermined threshold value for each of the at least one requirement of 


the printing process. 


13. The system according to any one of the claims 11 — 12, wherein the processing 
circuitry is further configured to: 


control the printing process based on the generated state-signal. 


14. The system according to any one of the claims 11 — 13, wherein the printing 
process comprises at least one of a pre-print phase, run-time print phase, and a post-print 


phase. 


15. The system according to claim 14, wherein the processing circuitry is further 
configured to: 
control the printing process by forming a continuation or interruption decision for each one 


of the phases of the printing process based on the determined state of the printing process. 


16. The system according to any one of claims 12 — 15, wherein the at least one 
requirement of the printing process comprises any one of an allowable calibration of a nozzle 
of the 3D bioprinter, an allowable priming of the nozzle of the 3D bioprinter, an allowable state 
of a dispensing needle connected to the nozzle of the 3D bioprinter, an indicator of clogging of 
the nozzle of the 3D printer, an allowable point tip calibration of the nozzle of the 3D bioprinter, 
an allowable flowrate and/or a volume, and/or a temperature, and/or a viscosity of a printing 
fluid being introduced into the nozzle of the 3D bioprinter, and an allowable size and/or a 
volume of a hanging droplet of the printing fluid extruded out of the tip of the nozzle of the 3D 


bioprinter and/or at least one quality-of-print metric. 
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17. The system according to any one the claims 11 — 16, wherein the plurality of 
sensor devices comprise any one of a camera, a thermal camera, a thermal sensor, a lidar, a 
radar, a laser, a Wheatstone bridge circuit arranged under a print bed of the 3D bioprinter, a 


pressure sensor, an ultrasound sensor, and a microwave radio frequency, RF, sensor. 


18. The system according to any one the claims 11 — 17, wherein the processing 
circuitry is further configured to: 
determine the weighted component, by means of the association weight function, by 


multiplying a predetermined weight value with each of: 

the at least one separately analyzed sensor data point, and/or 

the one or more analyzed-fused data point(s), and/or 

the one or more fused-analyzed sensor data combination(s), and/or 


the fused at least one separately analyzed sensor data point with the at least one of the one or 


more analyzed-fused data point(s). 


19. A 3D bioprinter comprising: 
one or more print heads, each print head comprising a dispensing nozzle having an inlet 
for introduction of a printing fluid into the dispensing nozzle and an outlet for extruding the 


printing fluid from the outlet of the dispensing nozzle; and 


a system according to any one of claims 11 — 18 for assessing the printing process by the 


3D bioprinter. 
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